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Abstract — In this paper we consider an integrated model 
for TCP/IP protocols with multipath routing. The model 
combines a Network Utility Maximization for rate control 
based on end-to-end queuing delays, with a Markovian Traf- 
fic Equilibrium for routing based on total expected delays. 
We prove the existence of a unique equilibrium state which 
is characterized as the solution of an unconstrained strictly 
convex program. A distributed algorithm for solving this 
optimization problem is proposed, with a brief discussion of 
how it can be implemented by adapting the current Internet 
protocols. 

I. Introduction 

Routing and congestion control are basic components 
of packet-switched communication networks. While rout- 
ing is responsible for determining efficient paths along 
which the sources communicate to their corresponding 
receivers, congestion control manages the transmission 
rate of each source in order to keep network conges- 
tion within reasonable limits. In current practice both 
mechanisms belong to separate design layers that operate 
on different time-scales: the IP layer (Internet Protocol) 
determines single-path routings which are updated on 
a slow time-scale, while the TCP layer (Transmission 
Control Protocol) corresponds to end-to-end users that 
perform rate congestion control at a faster pace for 
which routing can be considered to be fixed. Scalability 
considerations impose that these protocols must operate 
in a decentralized manner. 

Roughly speaking, TCP controls the rate of a source 
by managing a window size that bounds the maximum 
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number of outstanding packets that have been transmitted 
but not yet acknowledged by the receiver. Once this 
window size is reached the source must wait for an 
acknowledgment before sending a new packet, so the 
rate is approximately one window of packets per round- 
trip time. As the network gets congested, the round-trip 
time increases and the transmission rate is automatically 
slowed down. In addition, TCP dynamically adjusts the 
window size of a source in response to network con- 
gestion. To this end, links generate a scalar measure of 
their own congestion (e.g. packet loss probability, average 
queue length, queueing delay) and each source is fed back 
a congestion signal that reflects the aggregate congestion 
of the links along its route. This signal is used by the 
source to adjust its window size so that the higher the 
congestion the smaller the rate. The predominant TCP 
protocols in use are Tahoe and Reno which use packet 
loss as congestion measure, and Vegas which is based 
on queuing delay. We refer to 1 1 1 for a description and 
comparison of current protocols and their models. 

The interaction of many sources performing a decen- 
tralized congestion control based on feedback signals 
that are subject to estimation errors and communication 
delays, gives rise to complex dynamics that are difficult to 
analyze. However, assuming that the dynamics stabilize 
on a steady state, the equilibrium can be characterized as 
an optimal solution of a Network Utility Maximization 
(NUM) problem (see 0, 0, H). Thus, the TCP mech- 
anism can be viewed as a decentralized algorithm that 
seeks to optimize an aggregate utility function subject to 
network constraints. The NUM approach also allows to 
compare different protocols in regard to their fairness and 
efficiency. 



A second element of packet-switched networks is 
routing. This function is performed by routers in a 
decentralized manner using routing tables that determine 
the next hop for each destination. The routing tables are 
updated periodically by an asynchronous Dijkstra-type 
iteration that computes optimal paths according to some 
metric such as hop count, latency, delay, load, reliability, 
bandwidth, or a mixture of these. In current practice a 
single path that minimizes the number of hops is used 
for routing on each origin-destination pair. Research has 
also considered multipath routing strategies to improve 
performance by exploiting the available transmission ca- 
pacity on a set of alternative paths. The first paper on 
multipath routing was [5| which developed a distributed 
routing algorithm minimizing total delay in the case of 
flow dependent latencies but with fixed source rates. For 
the case of TCP rates that depend on network congestion, 
the model presented in [2] and further developed in |6), 
also considered a multipath routing but performed di- 
rectly by sources. Since distributed routing has scalability 
advantages, a router-based approach to multipath routing 
with elastic demands was investigated in J7] and [8|. 

In addition to increased efficiency, a further argument 
in favor of multipath routing is stability. Indeed, when 
route choice is based on metrics that are affected by 
congestion, such as queuing delay or link latencies, 
routing and rate control become mutually inter-dependent 
and equilibrium can be achieved only if both aspects are 
considered jointly: routing affects the rate control through 
the induced congestion signals, while rate control induces 
flows that determine in turn which routes are optimal. If 
routing is restricted to a single path, congestion effects 
may lead to route flaps. A remedy for such unstable 
behavior is to allow flows to split over multiple paths 
in order to balance their loads. An appropriate tool 
to capture these interactions between rate control and 
routing is provided by Wardrop equilibrium. On the other 
hand, since congestion metrics are subject to estimation 
errors and random effects it is natural to model routing 
as a stochastic equilibrium assignment. 

The goal of this paper is to propose an analytical 
framework that provides a theoretical support for cross- 
layer designs for rate control under multipath routing. Our 
model is close to the approach in Q, (8) and combines 
rate control modeled by NUM, with a routing strategy 
based on discrete choice distribution models that lead 
to a Markovian Traffic Equilibrium (MTE). The latter 



is a decentralized stochastic version of Wardrop's model. 
The combination of the NUM and MTE models leads to 
a system of equations that correspond to the optimality 
conditions of an equivalent Markovian Network Utility 
Maximization problem (MNUM), a strictly convex un- 
constrained program of low dimension where the vari- 
ables are the link congestion prices. This characterization 
allows to establish the existence and uniqueness of an 
equilibrium, and provides a basis for designing decen- 
tralized protocols for congestion control with multipath 
routing. 

The paper is structured as follows. Section §H\ reviews 
the basic components of our cross-layer approach: we 
recall the NUM framework for modeling the steady 
state of TCP protocols and we discuss the concepts 
of Wardrop equilibrium and Markovian routing. Section 



^ni] combines NUM and MTE, introducing the MNUM 
model for routing and rate control. In { III- A we reduce 



MNUM to a system of equations involving only the link 



congestion prices, and then in I III-B we show that these 
equations admit a variational characterization (D-MNUM) 
proving the existence of a unique equilibrium state. In 



{YV we briefly discuss how the model might lead to a 
cross-layer design of a distributed TCP/IP protocol. We 
close the paper with comparisons to previous work and 
some perspectives on future research. 

II. Notations and preliminaries 

The communication network is modeled by a directed 
graph G = (N, A), where the nodes i E N represent 
origins, destinations and intermediate routers, while the 
arcs a E A represent the network links. Each link is 
characterized by a latency A a = s a (w a ) = X a + ip a (w a ) 
where A° > represents a constant propagation delay 
and p a = 4)a{wa) is the expected queuing delay ex- 
pressed as a continuous and strictly increasing function 
tpa '■ [0, c a ) — > [0, oo) of the traffic w a on the link, with 
"0a (0) = and c a E (0, oo]. We also consider a finite set 
of sources k E K each one generating a flow rate x > 
from an origin Sk E N to a destination dk E N. 

A. Rate control and utility maximization under single 
path routing 

Suppose that each source k € K routes its flow along 
a fixed sequence of links (oi, . . . , Qj k ), so that the total 
traffic on a link a is w a — J2kBa xk where the summation 
is over all the sources k E K whose route contains 



that link. Consider the queuing delay p a = tp a { w a) as a 
measure of link congestion and assume that each source 
k £ K adjusts its rate x k as a decreasing function of 
the aggregate queuing delay q k = J2aekPa on i ts route, 
namely x k = fk{q k ), where fk : (0, oo) — > (0, oo) is 
continuous and strictly decreasing with fk{q k ) — > as 
q k — > oo. These equilibrium equations may be written as 
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1 k = y^P« =^2^a(w a ) 
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which correspond to the optimality conditions for the 
strictly convex minimization problem 



(NUM) min £* a (E s9 a^) 



a£A 



keK 



U k (x k ) 



where ^ a {-) denotes a primitive of ip a {-) and U k (-) a 
primitive of fZ {•). Alternatively, the equations may be 
stated in terms of the queuing delays as 



= ^fk{Y.bekPb) 



k3a 



k3a 

which correspond to the optimality conditions for the 
strictly convex dual problem 

rPa 
(D-NUM) minV / ii>- X {y)dy-y^F k {Y, bek Pb) 

P£R oTa J ° k^K 

where F k {-) is a primitive of /&(•). 

Example. Consider the model for TCP Vegas proposed 
in 0, H. For each source k and time t, let W k denote 
the size of the congestion window and T k = D k + q k the 
RTT expressed as the sum of the total propagation delay 
D k and the queueing delay q k . A Vegas source estimates 
D k as the minimum observed RTT in a time window, 
and tries to keep the difference between the expected rate 
x k = W k /D k and the actual rate x k — W k /T k close 
to a given value a k > 0. To this end, the congestion 
window is increased if x k —x k > a k , and decreased when 
x k —x k < a k . At equilibrium we must have x k —x k = a k 
which yields the equilibrium rate functions 



a k D k 



= fk(q k ). 



A simple model for the queuing delay can be obtained if 
we assume that each link has a service capacity c a > 
and an infinite buffer, so that an M/M/l model gives the 
expected queuing delay 



Pa 



H'n 



- i>a{w a ). 



C a {c a - W a ) 

The (NUM) formalism can handle other congestion mea- 
sures different from the queuing delay and has been used 
to model the steady state of different TCP protocols, each 
one characterized by specific maps /& and ip a (see Q), 

ED, ED, DS, HE), El)- 

B. Routing and traffic equilibrium 

We review next some equilibrium models for traffic in 
congested networks. In this setting the source flows x k 
are fixed but may be routed along a set of alternative 
paths R k connecting the origin s k to the destination d k . 
The basic modeling principle introduced by Wardrop in 
lfl4l is that at equilibrium only paths that are optimal 
should be used to route flow. We stress that, in contrast 
with rate control which is based solely on the queuing 
delays p a = ip a (w a ), route optimality will be measured 
using the total delays \ a = \^ + i> a {w a ). 

1) Wardrop equilibrium: Suppose that the flow x k 
is split into non-negative path-flows h r > so that 
x k = J2reB k hr, and let w a = J2r3a ^r be the induced 
total link-flows. Let H denote the set of such feasible 
flows (h, w). An equilibrium [14j is characterized by the 
fact that only optimal paths are used, namely, for each 
destination keK and each route r € R k one has 



h r > 



C r = Tfc 



(1) 



where c r = I] Q£r A a = J2 a er s a( w a) denotes the total 
delay of the route and r k = mm reR k c r is the minimum 
cost faced by source k. 

These equilibria were characterized in lfT31 as the 
optimal solutions of the convex program 



(P-W) 



min y 
(h,w)GH ^— ; 
aeA 



s a (z)dz. 



Since the feasible set H is compact this problem has 
optimal solutions, while strict convexity implies that the 
optimal w is unique. Alternatively, the equilibrium delays 
^a = Sa(wa) are the unique optimal solution of the 
strictly convex unconstrained dual problem 



(D-W) 
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Fig. 1. Variables for dynamic programming equations 



A„ 



where A° = s Q (0) and r fc (A) = min r6H * Xi aer • 
minimum total delay for source k £ If. 

2) Markovian routing and equilibrium: When link 
delays are subject to stochastic variability, the route 
delays c r become random variables and the equilibrium 
conditions (fill are replaced by a stochastic assignment of 
the form h r = x k P(c r is optimal). For instance, if the 
costs c r are i.i.d. Gumbel variables with expected value 
c r = E(c r ), we get the Logit distribution rule common 
in the transportation literature 



h r = . 



exp(— /3c r ) 



T, P eR" exp(-/3c p 



(Vrei? fc ) 



which assigns flow to all the paths, favoring those with 
smaller expected cost c r . The parameter j3 controls how 
concentrated is the repartition: for j3 ~ every path re- 
ceives an approximately equal share of the flow, while for 
j3 large the flow concentrates on paths with minimal cost. 
Unfortunately, such route-based distribution rules are not 
amenable to design decentralized scalable protocols. 

An alternative is to conceive routing as a stochastic 
dynamic programming process. Suppose that each packet 
experiences a random delay A a when traversing link a, 
and let f* be a random variable that represents the total 
delay from node i to destination dk. Denote A a = E(A a ) 
and 7j = E('?/ £ ) their expected values. If a packet at 
node i is routed through the link a £ A\ we have 
ff = A a + fj , so that a shortest path routing should 
choose the link with smallest A a + fj . Unfortunately, 
while the link delays A a for a £ Af might be observed at 
node i, this is not the case for the f,- 's which depend on 
future delays that will be experienced when traversing the 
downstream links. Suppose instead that only the expected 
values Tj are known and available at node i and that each 
packet from source k £ K observes the A a 's and is routed 
through the link a £ A^ that minimizes z k = X a + Tj 



AT 




At 



Fig. 2. Flow conservation diagram (here i = Sft) 



to the next node j a where the process repeats. Thus, 
is the denoting E* = {z% < z\ V6 € Af}, the packets from 
source k £ K move across the network according to a 
Markov chain with transition probabilities 



P 



P(£*) if ij = a 
otherwise 



(2) 



for i ^ dk, while the destination dk is an absorbing state. 
The expected flows correspond to the invariant measures 
of these Markov chains, leading to a flow distribution 
rule in which the throughput flow <\>\ from source k that 
enters node i, splits among the links a £ A\ according 
to (see Figure |2]l 



</>* m k a ) 



(3) 



The throughputs <fi k — (cj) k )i^d k can be computed from 
the stationary equations <fr k = J^TLoii^)']''^^' wnere 
P k — (P k j)ij^d k is me reduced transition matrix on 
the non-absorbing states, and S k — 1 for i — Sk 
and 8 k = otherwise. This may also be written as 
(f) k = x k 8 k + (P k )'4> k which corresponds to the standard 
flow conservation equations 



= x k S. 



kxk 



y 



6 a: 



(4) 



These equations can be restated in compact form using 
expected utility theory. Namely, let us write z k = z k + e k 
as the sum of its expected value z k = X a + r k a plus a 
noise e k with E(ejj) = 0, and assume that the distribution 
of e k does not change with z k (for a discussion of this 



assumption see I VI I. Then, the transition probabilities in 
(2 1 can be expressed as ¥(E k ) — -#jjr(z fc ) where tp k 
denote the expected utility functions 



rf{z k ) 



E(min{^ + e£}) if i / d k 

a£A+ 

if i = d h 



(5) 



which allow to rewrite the flow equations pj-pl as 

Vi ^ d k . 



'a — Yi ~d^k\ Z ) 

f>l = x k s k + E Q 



(6) 



e-AJ 



On the other hand, assuming that the cost-to-go variables 



{ 



~k 



£ A+} are independent from the local queueing 



times {A Q : a G A[ }, we may compute the expected 
value of f k by conditioning on the events E k as 



E(f, 



^/.■\ 



ZaeA+n^a 



ft 



E k )V(E k ) 



= EaeA+<*a + T k a \E k a )P(E k ) 



= E( min {A 



,}) 



so that 



- A„ + r„ fe 



Vi € AT 
Vae A. 



(7) 



Under mild conditions it was proved in lfl6l that, 
given the A a 's, system |6])-((7| has a unique solution 
(v, <j>, t, z). It was also shown that these equations, to- 
gether with the equilibrium conditions A a = s a (w a ) 
where w a — J2keK v a represents the total expected link 
load, have a unique solution (X,w,v,(f>,T,z) called a 
Markovian Traffic Equilibrium (MTE). This equilibrium 
is characterized by a pair of dual optimization problems 
analog to (p-w) and (D-w). As a matter of fact, the dual 
problem has exactly the same form 



(D-MTE) min 

Aeffi- 4 



E 

aS A 



A" 



ip a 1 (z)dz 



keK 



X k T k {\) 



where T k {\) = r k k (X) with T k (X) the solution of (m. 

The expected utility maps (p k (-) convey all the infor- 
mation required to describe a Markovian routing and may 
be considered as the primary modeling objects. These 
maps are determined by the random variables e k which 
are ultimately tied to the arc random costs A a . However, 
the class £ of maps that can be expressed in the form (J5]l 
admits an analytic characterization (see |16|): they are the 



C maps Lp : 



->• 



that are concave, componentwise 



non-decreasing, and which satisfy in addition 

(a) ip(xi + c, . . . , x„ + c) = ip(xi, . . . , x n ) + c 

(b) ip(x) — >• Xi when Xj — > oo for all j ^ i 

(c) for xi fixed, g^-(xi, . . . ,x n ) is a continuous dis- 
tribution function on the remaining variables. 



We remark also that <p(x) < min{a;i, . . . ,x n }. In what 
follows we assume the model is specified directly in terms 
of a family of maps ip k E £ with ip k d = 0, though they 
are not used explicitly by our distributed protocol in I IV 



Remark. Since packet movements are governed by a 
Markov chain, cycling may occur and additional con- 
ditions are required to ensure that packets reach the 
destination with probability one. A simple case is when 
source k considers only the arcs in Af that lead closer to 
destination d k {e.g. t_- < r*), so that the corresponding 
Markov chain is supported over an acyclic graph (N, A k ). 
To deal with this case it suffices to redefine 

¥>,V)=E( min {*£ + &) 



so that P k : 



i)z': 



(z k ) =0 for all a = ijgA k+ . 



III. Rate control with Markovian routing 

We proceed to develop a cross-layer model that com- 
bines a NUM approach for rate control based on queuing 
delays, with a Markovian multipath routing based on total 
delays. Each source k G K is characterized by an origin 
8k, a destination d k , and a continuous decreasing rate 
function f k : (0, oo) — > (0,oo) such that f k (q k ) -> 
when q k — > oo, while every link a E A has a continuous 
increasing latency function s a : [0, c a ) — > [A°,oo) with 
^■a = s a(0) > 0. Packets are routed according to a 
Markovian strategy characterized by a family of maps 
ip k € £ with ip\ = 0, and sources adjust their rates 
as a function x k = f k (q k ) of the total queuing delay 
q k = Tfc(A) — Tu, where r k (X) is the end-to-end expected 
delay as defined in the previous section and r° = T k (X°) 
is the free-flow value of this end-to-end delay. 

Informally, the source rates x k induce flows v k and 
total link loads w a . These loads determine link expected 
delays A Q = s a (w a ) that yield end-to-end delays Tfc(A) 
for each source and corresponding queuing delays q k . 
At equilibrium, these queuing delays must induce the 
original rates x k = f k (q k ). 

Definition 1: A pair (w,x) with w = (w a ) ae A and 
x = (x k ) ke K is called a Markovian Network Utility 
Maximization (MNUM) equilibrium iff w a — J2keK Ua 
where (v k , <p k ) solve the flow conservation constraints (pb 
with {r k ,z k ) satisfying (m), together with the link delay 
relations A a = s a {w a ) and the rate equilibrium conditions 
x k = fk(q k ) where q k = r fe (A) - t°. 



A. Reduced formulation of MNUM 

In order to establish the existence and uniqueness of 
equilibria we begin by reducing MNUM to an equivalent 
set of equations that involves only the variables A. To this 
end we need to extend the results in lfl6l for which we 
consider a fixed non-negative link delay vector (\ a )aeA- 
We first show that §fj uniquely defines z k and r k as 
implicit functions of A. This system can be equivalently 
stated solely in terms of the variables r k as 



* = Vi ((^a + ^.W) 



so it suffices to prove that the latter uniquely defines r k 
as a function of A. 

Proposition 1: Let k £ K and denote f k the cost of a 
shortest path from i to destination dk with link costs A a . 
Suppose also that f fc £ R^ is such that 



rt<^((K + ri) a&A ) (VieJV). 



Then f fc < r k and moreover, starting from r 
the iterates computed by 

k,n+l ,Jsi(\ 
T i = <Pi (\ X a + 



/,'.() 



l )aeA) 



(9) 



(10) 



are non-increasing and converge for n — > oo to a solution 
r k of ^ with r k e [f k , ?J°]. 

Proof: In order to prove that f fe < f let X = {i : 
T k < f k } and suppose by contradiction I ^ N. Since 
Tj < = fj we have d/. £ X. Consider a shortest 
path from a node i ^ X to dj., and let z ^ X be the last 
node before entering X and j £ X the next node. Since 

<P k ( zk ) ^ min aeA+ z q we § et 

T^ < d{{K+t k a ) aeA ) 



< Xi 



r k <\ 3 



Ta = Ti < T, . 



This contradiction proves that f fc < f . 

Let us prove next the convergence of the iteration ( fT0| ). 
We note that 



_fc,i 



P?((V 



_fc,0 



< min {A a + t 



)oeA) 



fe.O 



from which it follows inductively that the sequence (lOi 
is non-increasing: if r fc: " < r fe: " _1 then 



< 



^((Aa 






£AJ 



k,n— 1\ 



It remains to show that the sequence r fc: " is bounded 
below by f fc . We prove this by induction: the base case 
T k '° = f > f k was just proved above, while for the 
induction step it suffices to note that T k,n > f k implies 



fc,n+l 



> 



^ fe ((A„ 



k.n 



)aeA) 



< 



jtm 



■ :-A) > r t . 

By continuity it follows that the limit of T k ' n satisfies d8b 
as well as T k £ \T k ,f k ]. ■ 



(8) Remark. The previous result gives a procedure to solve 



8 p : compute the shortest path delays f and then iterate 
(nOb. Alternatively one may start from r fe0 = f fc in which 
case the iterates increase and are bounded from above by 
f k , hence these iterates also converge to a solution of (J8J). 

We present next an extension of a result from lfT6l 
which is the basis to prove uniqueness. In what follows 
we denote V the open convex domain of all A £ R A for 
which there exists T k £ R K satisfying 

T k < ip k {(X a + T k J aeA ) for all i ^ d fc . (11) 

Note that if A £ V then \' £V for all A' > A. 

Lemma 1: Let AeP and suppose that (r fe , z k ) solves 
(mi. Let Q k (z k ) be the matrix with entries Q k a (z k ) = 



-^(z k ) for i ^ dk and a £ A. 



Then 



(a) For each i ^ dk there exists j £ N such that Pfj, > 



and f fc 



3 ^ i 



(b) The matrix [I — P (z )] is invertible. 

(c) Equation (roji has a unique solution given by v k — 
Q k (z k )'4f>Q with 4> k = [I-P k (z k )']- l 8 k x k >Q. 

Proof: (a) Since the ip k, s are concave and differen- 
tiable we have 

tf((Xa+T k a ) aeA ) 

tf(**) + o E^£(**)(?£ 

t^+ E^(^-^) 



< 
< 



^) 



from which (a) follows directly. 

(b) Given i ^ dk and using (a) inductively we can find a 
finite sequence of nodes i$, . , . , i m with io =i, i m = dk 
and Pi 



k^k + l 



)a£A) 



> 0. Thus, starting from i, the chain has a 
positive probability of reaching the absorbing state du in 
a finite number of steps. This implies that p k ( yZ k ) m is 
strictly submarkovian for m large enough, and therefore 

[I - P k (z k )} is invertible. 



(c) The first equation of (rob can be rewritten as v k = 
Q k (z k ) <p k , which substituted into the second equation 
yields (f) k = S k x k + P k (z k )'(j> k , so that (b) implies 
4> k = [I — P k (z k ) ]~ 1 5 k x k . The non-negativity of these 
quantities follows from the fact that x k > while the 
matrices Q k (z k ) and P k (z k ) have non-negative entries 

With [I - P^)']- 1 = Enr=olP k (z k )'] m - ■ 

The next result is the key to reduce the MNUM 
equations to a system in the variables A. 

Proposition 2: If A E V then, for each source k G K, 
the system (m) has a unique solution z k = z k (X) > 
and r k — T k (X) > 0. Moreover, the functions A — > r k (X) 
and A — > z k (X) are concave, smooth and component-wise 
non-decreasing. 

Proof: It suffices to show that <[8j defines implicit 
maps A H> T k (X) over the domain V which are well 
defined, concave, smooth, and monotone. We already 
proved the existence of a solution with r k > r k . Let 
us prove its uniqueness. 

Uniqueness: Let k € K and consider two solutions 
(r 1 , z 1 ) and (r 2 , z 2 ) for (|7j. Let a — m&x i( z N (T 2 — rj) 
and denote N* the set of nodes where the maximum is 
attained. For every i € N*, the concavity of ip k (■) gives 

rf = V > k (z 2 ) < vK^+lZaeAi'&S^l-zl) 

and, since (r? — rj) < a while the partial derivatives 
add up to 1, we get 



T 2 - T. 



!<E a ^ d ^t^)(rl-rl)<a. 



It follows that for every a € Af such that gfHz 1 ) > 
we necessarily have j a £ N* . Combining this fact with 
Lemma ITTa), we can find a finite sequence of nodes in 
N* starting at i and ending at d^. Hence dk € ^V* so that 
a = rj — t\ =0, which implies t 2 < t 1 . Exchanging 
the roles of t 1 and r 2 we get the converse inequality so 
that t 1 = t 2 proving uniqueness. 

Concavity: For a<E (0, 1) set X a = a\ 1 + (l-a)A 2 and 
T a = aT k (\ 1 )+{l-a)T k {\ 2 ). Denote z\ = Xl + r^X 1 ), 



rf. Then z a 



X 2 a + r k a (A 2 ), and z° = A" 
az 1 + (1 — a)z 2 and the concavity of ip k implies 

tfiz*) > av k (z l ) + {l-a)tf{z 2 ) 
= arHX^ + il-a^iX 2 ) 

= T a . 



This proves that f k — r a satisfies condition J9I) for A", 
and then Proposition n] gives T k (X a ) > r a which yields 
precisely the concavity inequality 

^(A^ar^ + U-^A 2 ). 

Smoothness: This is a direct consequence of the implicit 
function theorem. Indeed, noting that Tj — we may 
reduce (J8J) to a system in the variables (r k )i^d k - The 
Jacobian of this reduced system is [/ — P k ] which is 
invertible by Lemma |TJb), so the conclusion follows. 

Monotonicity: Let A 1 G V and take A 2 > A 1 so that 
A 2 e V. The monotonicity of ip k implies 

r^A 1 ) = ^((Ai + rJKA 1 ))^) 

< v k {{X 2 a + r k a (X l )) aeA ) 

Hence r k = r fc (A 1 ) satisfies J9j) for A 2 , and Proposition 
[J] implies T fc (A 2 ) > ^(A 1 ). ' ■ 

The implicit maps r k (X) and z k (X) defined by (m), 
allow to restate the MNUM equations solely in terms of 
the link delay vector A. Indeed, let q k (X) = r k k (A) — t£ 
and define i k (X) — fk(q k (X)). According to Lemma 
me) the equations (roll have unique solutions v k = v k (X) 
and 4> k = (j) k (X). Denoting w a (X) = J2keK v a( x )^ the 
MNUM equations are equivalent to the reduced system 
of equations 



(R-MNUM) 



K = s a (w a (X)) yaeA. 



B. Variational characterization 

We show that the reduced system (R-MNUM) corre- 
sponds to the optimality conditions of an optimization 
problem which is a combination of the variational char- 
acterizations (d-num) and (d-mte). 

Theorem 1: Assume that A E V. Then (x*,w*) is 
an MNUM equilibium iff x* — £(X*) and w* — w(X*) 
where A* is the unique optimal solution of the strictly 
convex program 
(d-mnum) 

min $(A) 4 J2 f Xa s- a 1 {z)dz-Y J Fk{q k {X)) 



xev 



aeA 



-V! 



keK 



(A) — t° is concave 



with Fk(-) a primitive of /fe(-). 
Proof: Since q k (X) = t 
and the /^'s are positive and decreasing, it follows 
that the map < I > i(A) = — 2~2keK ^k(q k W) i s convex. 
Also, since the s^'s are increasing we obtain that 



<I>o(A) = 2~2 f\o s ~^ 1 { z )dz is stricty convex, and then 

so is $(A) = $o(A) + $i(A). Hence, since "P is 
open and convex, an optimal solution for (D-MNUM) is 
characterized by V$(A) = 0. Now 



f)\ a 



SaHK) 



£ Mi k W)^W 



k£K 



e £ fc (A) ^e(A) 



fcS-R" 

An implicit differentiation of A81 gives — 



dr k 
dX 



P k (X) 

which we get 



so that 



dr k 



<)\ 



Q(A) 



[I - P fe (A)]- 1 g(A)e a , from 



dr. 



x k (X)^(X)=x k (X)e' Sk [I-P k (X)]-'Q(X)e a 
= ([I-P k (X)'}- 1 S k x k (X))'Q(X)e a 
= <j> k {X)'Q{X)e a 
= v k (X)'e a 
= v k a (X). 

Therefore ■£?- — s~ 1 {X a ) — w a (X) and the optimality 
condition V$(A) = coincides with the (R-MNUM) 
characterization of equilibria. ■ 

This characterization allows us to prove the existence 
and uniqueness of an MNUM equilibrium. 

Theorem 2: Problem (D-MNUM) is strictly convex and 
coercive, hence it has a unique optimal solution and 
therefore there exists a unique MNUM equilibrium. 

Proof: We already showed that the objective function 
$(A) is strictly convex. Moreover, since q k (X) is com- 
ponentwise non-decreasing while the term J^J* s~ 1 (y) dy 
is decreasing for A a < A° , it follows that the minimum 
of (D-MNUM) belongs to the set {A > A }. Hence, in 
order to establish coercivity, it suffices to show that the 
recession function satisfies $°°(A) > for all A > 
with A ^ 0. Now, 



$o°°(A) 



lim -$ (tA) 

t— >oo t 



I rtX a 

Yl A m 7 / S a X ( Z ) dz 
f"^ *^°° t J X o 



a£A 



a£A 



lim s Q 1 (U Q )A a 



Y^ C a X a > 0. 



In order to compute $J°(A) let us consider the convex 
maps Uk(r) = ~Fk(—r) and hk(X) = —q k (X). It can be 
shown (see lfl6l ) that hff?(X) — —f k with f k the cost 
of a shortest path with link delays A a . Since U£°(r) = 
for r < 0, we get (U k o /i fc )°°(A) = U%°(hf(X)) = 
for all A > 0, and then $J°(A) = 0. As a consequence, 
for all directions A > with A^Owe get $°°(A) > 0, 
so that (D-MNUM) is inf-compact and therefore it has an 
optimal solution. ■ 

IV. A DISTRIBUTED ALGORITHM FOR MNUM 

This section describes how the MNUM framework 
can lead to a distributed protocol for rate congestion 
control under Markovian multipath routing. This protocol 
can be interpreted as a distributed algorithm that solves 
the variational problem (D-MNUM). The algorithm is 
based on a Markovian routing process for packets, with 
a slow update of the end-to-end expected delays T k 's. 
This process is combined with a fast TCP adaptation of 
user's rates by estimating the end-to-end queuing delays 
to reach the equilibrium rates x k (X). A more detailed 
description and analysis of the distributed protocol will 
be the subject of a forthcoming paper ifTTIl . 

A. Packet routing based on local queues 



a£A 



We adapt the ideas of SII-B2 in order to define a 
routing policy based on local information. To do this, 
router i needs to observe the values z k — A a +r- of each 
outgoing link a G A\ by adding the link propagation 
delay A°, plus the current queueing delay p a of the link, 
plus the estimate of the expected delay r,- informed by 
the next hop. 

Routers must periodically provide their neighboring 
routers with an estimate of the total expected delays r* . 
These estimates may be updated on a slow time-scale 
by averaging the observed delays mm aeA +{X a + Tj} 
over all packets routed on a fixed time window or a fixed 
number of packets. The observed average f k is used to 
update the estimate of the expected delay as 

T k ^{l-a)T k + af k . 

B. TCP protocol 

A TCP protocol for source rate control requires a con- 
sistent feedback congestion signal. Basically, we require a 
mechanism by which every source k can estimate q k (X). 
Here we rely on the two time-scales assumption: sources 
control their rates at a much faster pace than routers, so 



they see link delays as constant. For fixed expected delays 
A, the expected forward time coincides with tt (A), thus 
using standard protocols for estimating the forward time, 
users can get an unbiased estimation T k of this total 
expected delay. 

Finally, we need to compute the free-flow total ex- 
pected delay r?. Since the maps (f k (-) are not known 
explicitly, it is not possible to solve the equations <JVj 
directly. We propose to estimate r° by the minimum 
observed time for every packet, as in the single-path 
implementation of Vegas. This provides a lower bound 
estimate for r°. 

The free-flow times t® together with the unbiased 
estimators T k of r k for every packet t arriving to 
destination, can be used to adjust the rates by a stochastic 
approximation algorithm of the form 



■-t+i 



(l-5)x k +Sf k (Q k ) 



(12) 



where Q k = T k — t®. If we let sources adapt long enough 
so that Q k ~ q k and x k ~ fk(q k ), we can then proceed 
to update the router estimates of the end-to-end delays rf. 
Further considerations on how multipath routing affects 
buffering at the destination and other issues will be 
addressed in ifTTl . 

V. Comparison with previous work 

As mentioned in the Introduction there have been 
several proposals to develop multipath routing protocols, 
such as 0, and |6|. However, our work is more 
closely connected to Q, (8) which describes an approach 
for rate congestion control under a distributed multipath 
routing protocol based on routing proportions (a k ) aeA + 
that control the routing of packets from source k at node 
i. These proportions are dynamically adjusted so that 
eventually the routing concentrates over the links a e A\ 
that belong to shortest expected paths. 

From a conceptual point of view there are two main 
differences between this approach and ours. Firstly, the 
proportion-based approach makes routing decisions based 
on expected values, while our splitting of traffic evolves 
stochastically as it uses the current state of local queues 
to optimize the routing. A second difference is that Q, 
|8| optimize routing in terms of queuing delay only, 
while our routing optimization considers the total delay 
including queuing plus propagation delay. This choice 
makes sense if one cares about the total time that packets 



take to be transmitted, and not just the time spent in 
queues. 

Further differences arise at the implementation level. 
The protocol in Q, (8) requires three time-scales: a fast 
TCP rate adaptation, a medium time-scale for route price 
updates, and a slow update for flow-splitting proportions. 
This time-scale separation is required to justify the con- 
vergence of the protocol. In our case we only need two 
time-scales: a slow one for estimating the delays and a 
fast one for TCP rate control and routing. 

Finally, in terms of information and communication 
overhead both implementations are similar, with the dif- 
ference that in our case we require a fast interaction with 
local queues in order to choose the best route. 

VI. Conclusions and future work 

We proposed a new cross-layering model for TCP/IP 
control under multipath routing. The motivation for our 
routing mechanism comes from using local information 
about queueing delays as well as the expected delays 
from the next hops to the destination, in order to exploit 
the available capacity by sending packets through several 
alternative routes. To achieve this purpose, we considered 
a Markovian routing combined with a Vegas-like TCP 
protocol for rate control. The routing process was char- 
acterized by studying the expected dynamic programming 
equations which lead to a Markovian Traffic Equilibrium, 
together with a standard Network Utility Maximization 
model for the TCP steady state. This led to a variational 
characterization of the equilibrium that allowed to prove 
its existence and uniqueness, and which inspired a dis- 
tributed protocol for attaining the equilibrium. 

There are several unsolved issues. Firstly, further re- 
search is required to provide a theoretical support for 
the convergence of these protocols. A detailed analysis 
should study the relation between the packet-level dy- 
namics and our flow-level model. Our equilibrium model 
relies on this assumption: namely, we base our updates 
on aggregated flow information as well as in the two 
time-scales convergence of equilibrium flows. A complete 
analysis should explain to which extent the flow model 
captures the packet level dynamics, and how fast the 
equilibrium flows are attained by sources. Interesting 
recent results along this line can be found in ifTSl . |[T9l . 

Another interesting question is related to the model of 
randomness assumed. We considered an additive structure 
z k = z k + e k which presumes the same variability of 



delays regardless of the average flow levels observed. A 
more realistic model should consider higher variability 
for higher expected delays, based either on a detailed 
analysis of the distribution of waiting times at queues, 
or at least using a simplified multiplicative randomness 
model of the form z\ = z%(l + e„). 

A final line of research has to do with simulating 
this protocol in a realistic environment. A fair com- 
parison with single-path routing requires the presence 
of uncertainty and delays in information transmission. 
Simulation may provide an idea on the effective increase 
in performance that one might expect from a Markovian 
multipath routing. 
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